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1.0  SUMMARY 


In  the  context  of  the  overall  Warfighter  Interface  Research  and  Technology  Operations  (WIRTO) 
research,  this  task  focused  on  supporting  research  in  cognitive  modeling  and  dynamic 
visualization  of  understanding.  Models  of  tacit  knowledge  need  to  be  visualized  in  a  format  that 
provides  analysis,  comparison,  manipulation,  and  recognition  of  that  knowledge.  Research  into 
data  collection  tools  and  translation  services  was  executed.  Results  are  provided. 
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2.0  INTRODUCTION 


The  Polyhedral  Dynamics  Analysis  Tool  (PD AT)  was  modified  to  incorporate  scenario  data  and 
analysis.  Scenario  data  is  generated  from  scenario  runs  using  OMAR  (Operator  Model 
Architecture)  and  Neverwinter’s  Night  (NWN).  PD  AT  will  provide  tools  to  analyze  knowledge 
models  and  recognize  knowledge  patterns.  Research  into  data  collection  tools  and  translation 
services  was  executed. 


Distribution  A:  Approved  for  public  release;  distribution  unlimited.  88ABW-2012-3864,  13  July  2012 


3.0  METHODS,  ASSUMPTIONS  AND  PROCEDURES 

3.1  Scenario  Generation  and  Display 

Scenarios  are  run  using  Operator  Model  Architecture  (OMAR)  to  send  commands  to 
Neverwinter’s  Night  which  displays  the  scenario  in  a  virtual  environment. 

3.1.1.  OMAR 

OMAR  (omar.bbn.com)  is  a  software  suite  that  supports  the  development  of  simulation  and 
agent-based  systems.  OMAR  is  the  driver  for  the  scenarios  that  have  been  developed.  The  two 
main  functions  are  to  send  and  receive  data  on  the  scenario  to  NWN,  and  to  output  the  results  of 
the  scenario  into  a  scenario  (.sen)  file  for  use  in  the  PD  AT  application. 

3.1.2.  NWN 

NWN  is  a  software  application  that  provides  a  virtual  environment  for  our  scenarios.  Virtual 
environments  have  been  generated  in  the  application  to  simulate  our  generated  scenarios.  Agents 
are  controlled  via  command  from  the  OMAR  application,  and  results  of  the  actions  are  returned 
to  OMAR  for  recording.  Figures  1,  2,  and  3  have  examples  of  the  office  virtual  environment  that 
was  generated. 


Figure  1:  NWM 
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Figure  3:  NWN  Conversation 


3.2  Analysis  Application 

Analysis  of  the  scenario  is  performed  by  the  PDAT  application. 

3.2.1.  PDAT 

PDAT  was  updated  to  handle  scenario  data  that  come  from  the  OMAR/Neverwinter  Night’s 
scenarios. 

3.2.2.  Scenario  Tools 

In  order  to  fully  use  the  data  provided  in  the  scenario  files,  there  were  a  number  of  scenario  tools 
that  were  added  for  use  in  PDAT.  These  tools  include  Run  Scenario,  Manual  Scenario,  Single 
Session  Comparison,  Dual  Session  Comparison,  and  Dual  Side  By  Side  Comparison. 

3.3  Researching  Categorization  and  Stereotyping 

Research  was  conducted  to  expand  the  scenarios  with  better  knowledge  models. 

3.3.1.  Data  Collection 

A  collection  tool  was  needed  to  gather  data  for  future  knowledge  models. 

Criteria  for  data  collection  tool:  A  variety  of  criteria  was  used  to  select  the  software  used  for 
data  collection.  They  included: 

•  Response  Count 

•  Data  Format 

•  Languages  Supported 
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•  Software  documentation 

•  Extensible 

•  Clean  display 

•  Software  support 

•  Cost 

The  data  collection  software  was  evaluated  against  these  criteria.  Any  tool  that  didn’t  meet  the 
first  three  criteria  was  thrown  out,  as  it  would  be  unable  to  accomplish  the  tasks  required.  Once 
a  tool  met  these  main  requirements,  it  was  tested  on  how  well  it  met  the  other  requirements  and 
how  difficult  it  would  be  to  overcome  any  deficiencies. 

Data  Collection  Options:  In  all,  24  data  collection  tools  were  evaluated  for  potential  use.  Seven 
were  selected  for  final  analysis  and  17  options  were  discarded  for  failing  to  meet  a  required 
specification.  The  17  discarded  tools  were  Digivey,  Fluid  Surveys,  Inquisite,  Survey  Systems, 
Survey  Pro  5,  Global  Park,  Cvent,  Blue/Surveys,  Checkbox  Survey  Online,  Checkbox  Survey 
Server,  Cogix  ViewsFlash,  Feedback  Server  On-Demand,  Feedback  Server  On-Site,  KeySurvey, 
phpESP,  StatPac,  and  Survey  Said.  The  seven  selected  tools  are  detailed  in  the  following 
sections. 

Lime  Surveys:  Time  Surveys  is  a  free  on-line  survey  package  with  unlimited  surveys  and 
responses.  It  has  adequate  data  output  formats  and  can  prevent  someone  from  taking  the  survey 
multiple  times.  The  tool  is  basic,  and  it  may  not  cover  all  the  languages  needed  for  this  research. 
The  question  types  are  limited. 

See  URF:  http://www.limesurvey.org 

Qualtrics:  Qualtrics  is  an  on-line  survey  that  provides  a  large  number  of  features  including  an 
extensive  question  type  selection,  language  support,  and  an  on-line  library  of  tips  and  support. 
Even  with  purchase,  there  is  a  limited  number  of  responses  allowed,  and  only  one  user  may  edit 
the  surveys. 

See  URF:  http://www.qualtrics.com/survey-software/ 

Snap  Surveys:  Snap  Surveys  provides  two  options,  a  survey  hosted  on-line  or  a  downloadable 
survey  to  host  on  your  own  server.  It  covers  the  languages  needed  and  has  acceptable  output 
formats.  The  cost  is  high  in  comparison  to  the  quality  of  the  surveys,  and  the  software  does  not 
provide  any  time  controls. 

See  URF:  http://www.snapsurveys.com/us/ 

Survey  Gizmo :  Survey  Gizmo  provides  usable  data  output  formats  along  with  unlimited  surveys 
and  language  support.  Timing  controls  are  lacking,  and  does  not  provide  translations. 

See  URF:  http://www.surveygizmo.com/ 
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Survey  Monkey  Pro:  Survey  Monkey  Pro  is  an  application  that  was  recommended  by  co¬ 
workers  who  currently  use  it.  It  provides  1000  responses  at  a  low  monthly  cost.  It  is  also  easy  to 
use.  It  provides  great  service  for  the  basic  features,  but  is  not  extensible  enough  to  provide  the 
controls  needed  for  the  research. 

See  URL:  http://www.surveymonkey.com 

Toluna:  Toluna  is  a  company  which  has  a  basic  survey  tool  with  few  features.  Toluna  may  be  a 
great  resource  in  panel  procurement  if  needed  in  the  future.  The  lack  of  features  make  this  tool 
unacceptable  for  our  use. 

See  URL:  http://www.toluna-group.com 

Zoomerang:  Zoomerang  provides  unlimited  surveys  and  responses.  It  can  display  images,  but 
not  video.  It  can  be  used  in  a  kiosk  version  and  provides  adequate  data  output  formats.  It  cannot 
control  time.  It  also  does  not  support  all  the  languages  we  need,  and  only  allows  for  one  user. 

See  URL:  http://www.zoomerang.com/online-surveys/ 

3.3.2.  Translation 

The  surveys  will  be  conducted  in  a  number  of  different  languages,  so  a  translation  tool  will  be 
needed. 

Criteria  for  Translation  Tool:  A  variety  of  criteria  were  used  to  select  the  software  for  data 
collection.  The  criteria  included: 

•  Languages  Supported 

•  Human  Translated  vs.  Machine  Translated 

•  Amount  of  time  required  for  translation 

•  Cost 

The  data  collection  software  was  evaluated  against  these  criteria.  The  four  language  translations 
that  the  software  must  provide  are  Chinese,  Urdu,  Sanskrit  and  Hindi.  Human  translations  will 
be  better  for  sentences,  but  machine  translation  may  be  adequate  for  some  of  the  one  word 
responses. 

To  receive  an  adequate  quote  on  cost  and  time,  the  number  of  words  requested  was  1000  for  each 
of  the  required  languages. 

Translation  Options:  There  were  six  evaluated  services  that  provide  human  translation  of  the 
four  languages,  and  three  machine  translation  tools  that  provide  translations  for  three  of  the 
languages  (sans  Sanskrit).  A  breakdown  is  shown  in  Figure  4. 

Human  Translation  Options:  The  human  translation  options  are  Advanced  Language, 

AltaLang,  Betmar,  Lionbridge,  Transperfect,  and  Verbatim  Solutions. 

Machine  Translation  Options:  The  machine  translation  options  are  Ace  Translator,  Babylon 
and  Google. 
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Figure  4:  Translation  Option  Table 
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4.0 


RESULTS 


4.1  PD  AT  Data  Generator 

The  PD  AT  Data  Generator  takes  the  scenario  files  generated  from  OMAR  and  parses  out  the 
applicable  data  for  use  in  generating  a  backcloth  (.bclth)  file  for  use  in  PD  AT.  The  backcloth 
lists  nodes  that  are  elements  of  the  scenario  and  places  them  on  the  graph  according  to  the  type 
and  time  of  the  event.  The  Generator  can  also  create  backcloth  files  without  loading  a  scenario 
file,  but  the  output  is  less  sophisticated  and  more  time  intensive. 

4.1.1.  Main  Screen 

The  main  screen  has  three  major  sections,  scenario  panel,  nodes  panel,  and  the  concepts  panel. 
The  scenario  panel  is  used  to  load  the  scenario  file.  The  concepts  (agents)  can  be  selected  on 
load.  Once  loaded,  the  nodes  and  concepts  are  filled  in.  The  lists  can  be  modified  in  each 
respective  panel.  Once  the  data  is  correct,  the  user  saves  the  data  into  a  backcloth  file. 

4.1.2.  Backcloth  File 

The  backcloth  file  is  generated  by  the  PD  AT  Data  Generator,  and  loaded  by  PD  AT.  The  file  is 
xml  based,  and  it  contains  the  nodes  and  concepts  that  will  be  graphed.  Along  with  the  nodes 
and  concepts  is  the  scenario  data.  The  scenario  data  lists  nodes  and  concepts,  along  with  the 
time  at  which  they  are  activated.  The  order  of  the  nodes  also  determines  the  path  that  is  graphed. 
The  node  and  concept  list  contain  all  the  possible  activation  nodes  and  concepts,  but  each 
scenario  within  the  file  may  use  a  smaller  percentage  of  them. 

4.2  PDAT 

4.2.1.  Main  Screen  Display 

The  main  screen  (Figure  5)  is  made  up  of  five  elements:  Menu,  Tool  Bar,  Graph,  Data,  and 
Status  Bar.  This  design  has  the  main  elements  displayed  (graph,  concept,  and  node  lists)  with 
other  features  one  to  two  clicks  away. 
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■Jt-  Polyhedral  Dynamics  Analysis  Tool  (JaneTest)' 


File  View  Reports  Scenario  Options  Layout  Help 


Figure  5:  PDAT  Application 


4.2.2.  Menu 

The  menu  (Figure  6)  has  seven  menu  items:  File,  View,  Reports,  Scenario,  Options,  Layout,  and 
Help.  The  File  menu  contains  the  file  load/save  actions  for  the  application.  The  View  menu  has 
viewing  selections  for  the  application.  The  Reports  menu  list  reports  for  the  applications,  broken 
up  in  four  sections:  Graph  Data,  Node,  Edge  and  Concepts.  All  scenario  tools  are  found  under 
the  Scenario  menu.  Application  options  are  in  the  Options  menu.  The  different  layouts  in  which 
the  graph  can  be  generated  are  in  the  Layout  menu.  The  Help  menu  contains  the  various  help 
features. 
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Figure  6:  Menu 

4.2.3.  Tool  Bar 

The  tool  bar  (Figure  7)  has  quick  launch  icons  for  the  application.  It  also  contains  some  graph 
manipulation  tools,  such  as  zoom  and  node  sizing.  The  quick  launch  icons  contain  some  of  the 
menu  items.  The  last  element  is  the  Concepts  combo  box. 


□  E?H  I  +  -SI  ‘=Ck 


Concepts 

All 


Figure  7:  Tool  Bar 


4.2.4.  Graph 

The  majority  of  the  application  display  is  the  graph  itself.  The  graph  can  be  built  in  two 
different  ways,  Polyhedral  and  two  dimensional  (2D)  (Figure  8).  The  polyhedral  will  be  a  three 
dimensional  display  of  the  graph.  The  2D  is  a  two  dimensional  display  of  the  graph.  2D 
displays  typically  generate  faster  than  polyhedral  displays. 
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Figure  8:  Polyhedral  Graph  vs.  2D  Graph 


4.2.5.  Data 

The  right  side  of  the  application  contains  the  graph  data.  It  is  split  into  two  lists:  concepts  and 
concept  nodes.  The  concept  list  has  all  concepts  that  are  used  to  build  the  graph.  The  concept 
node  list  populates  with  data  when  a  concept  is  selected.  The  lists  provide  some  functionality, 
mainly  centering  selected  nodes  and  highlighting  selected  concepts.  Concepts  can  also  be 
modified  through  this  display. 

The  right  side  of  the  application  can  display  reports/tools,  depending  on  whether  reports  are 
generated  as  pop-ups  or  tabs  on  the  right  side. 

4.2.6.  Status  Bar 

The  status  bar  sits  at  the  bottom  of  the  application.  It  displays  information  and  warnings  about 
the  application.  Information  is  displayed  for  a  given  amount  of  time  and  displays  information 
such  as  node  data  and  load  status.  Errors  display  in  red,  without  a  timer,  and  inform  the  user  of  a 
failure  in  the  application,  such  as  a  file  that  did  not  load  correctly. 

The  status  bar  also  displays  progress  for  those  functions  which  are  time  consuming,  such  as 
drawing  a  large  graph,  or  loading  data. 

4.2.7.  Scenario  Tools 

The  Scenario  menu  contains  six  different  tools  for  the  application.  These  tools  are  similar  to  the 
reports;  they  can  be  displayed  as  a  tab  or  in  their  own  window.  The  scenario  tools  provide  two 
key  functions:  displaying  graph  activation  from  the  provided  scenario  and  analysis  of  the 
scenario. 
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Run  Scenario :  Run  Scenario  (Figure  9)  is  an  application  that  takes  Scenario  files  (.sen)  that 
were  generated  by  OMAR  and  allows  the  user  to  run  the  scenario  against  the  backcloth  loaded 
into  the  PD  AT  graph.  Run  Scenario  has  a  number  of  settings  for  manipulation  of  how  the 
scenario  is  run  and  viewed.  These  settings  include:  following  scenario  time,  or  specifying  the 
amount  of  time  between  each  event.  Nodes  are  activated  in  the  graph  according  the 
specifications  of  the  scenario,  and  the  amount  of  time  the  node  is  activated  can  be  modified. 

Run  Scenario  can  also  be  modified  to  take  images  for  future  analysis.  Images  can  be  predefined 
in  the  settings,  such  as  after  an  activation  event  or  in  a  constant  interval.  Image  events  can  be 
added  on  the  main  screen  for  a  specific  time  or  event  number. 

The  application  is  broken  up  into  six  parts:  Session,  Scenario,  Show  Node,  Event,  Activation  and 
Button  Panel.  Session  contains  the  session  information,  name,  time,  nodes  activated  and  concepts 
activated.  Scenario  loads  the  scenario  file.  Show  Node  sets  the  node  for  display  in  the  activation 
panel.  Event  is  broken  into  three  tabs:  All,  Recent  and  Imaging.  All  lists  all  events  in  the 
scenario.  Recent  lists  the  last  event,  the  current  event,  and  the  next  two  events,  with  the  current 
event  larger  for  easy  viewing.  Imaging  lists  and  modifies  imaging  events.  The  activation  panel 
displays  a  line  graph  for  Interest  nodes  and  attribute  and  disposition  values  for  decision  nodes. 
The  button  panel  has  the  settings  and  scenario  run  controls. 

The  application  will  run  through  the  scenario,  displaying  each  event  as  it  is  processed,  and 
updates  the  graph  and  activation  panel.  Images  are  taken  and  stored  in  the  file  system  for  later 
analysis. 
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Figure  9:  Run  Scenario 
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Manual  Exercise:  Manual  Exercise  is  a  simple  exercise  tool  that  uses  data  that  is  completely 
user  generated.  The  user  creates  a  list  of  attributes  to  activate  and  sets  the  level  of  activation. 
Once  the  list  of  activations  are  created  and  applied,  the  matching  concepts  are  listed  in  order  of 
closest  match.  This  tool  can  be  used  in  defining  stereotypes  based  on  the  loaded  backcloth.  (See 
Figure  10.) 


Figure  10:  Manual  Exercise 
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Session  Comparison:  Session  Comparison  takes  the  session  data  saved  from  Run  Scenario  and 
compares  it  to  a  second  set  of  session  data.  The  display  is  broken  up  into  three  sections:  a  session 
panel,  a  common  panel  and  another  session  panel.  Each  session  has  its  own  display,  with  the 
common  panel  containing  common  data  in  between  the  two.  The  session  panel  displays  a  graph 
with  lists  of  unique  activated  nodes  and  concepts.  The  graph  and  lists  are  pulled  from  the  time  or 
event  selected  in  the  common  section.  If  an  activated  node  is  activated  in  both  sessions,  the 
information  is  placed  in  the  common  area.  (See  Figure  11.) 


Session  Comparison 


Session 

Time:  00:1 

Common  Data 

Graph  Session  Comparision 

□BB 


::  00:17 


Unique  Nodes  Activated 


Unique  Concepts  Activated 


Common  Nodes  Activated 

X2 :  call-niece-on-concern-f or-daughter-sunni 

X6:  religion-husband 

X7;  religion-agent 

X9:  Selecting  call-niece-on-concern-for-daughter-  procedures. 
X10:  says  (Asalaam  Alaikum  Naheed.) 

X13:  picks  up  cellphone. 

X16:  initialize  Dearborn 

XI 7:  initialize  RashidHome 

X20:  call-niece-about-daughter 

X22:  religion-daughter 

X23:  attended  (Asalaam  Alaikum  urn.) 

X25:  dials  Lateefa. 

X26:  call-niece 


Common  Concept  Activated 


Unique  Nodes  Activated 


Unique  Concepts  Activated 


Figure  11:  Session  Comparison 
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Single  Session  Run :  Single  Session  Run  is  very  similar  to  Run  Scenario,  in  that  it  takes  a 
scenario  file  and  activates  the  graph  based  on  events  from  the  scenario  file.  It  differs  in  that  it 
saves  no  session  or  image  information  and  has  no  activation  panel.  The  main  difference  is  that  it 
does  not  run  through  the  scenario  in  a  step-by-step,  time-based  manner.  The  user  selects  a  range 
of  time  or  events  (i.e.,  from  1  minute  to  3  minutes,  or  event  6  through  event  12),  and  all 
nodes/concepts  that  are  active  in  this  range  will  be  activated  immediately.  (See  Figure  12.) 


Figure  12:  Single  Session  Comparison 
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Dual  Session  Comparison:  Dual  Session  Comparison  is  similar  to  Single  Session  Run,  but  uses 
two  scenarios  instead  of  one.  Like  Single  Session,  the  scenarios  are  loaded,  the  ranges  are 
selected  and  the  graph  is  updated  to  display  the  activations.  Both  scenarios  are  displayed  in  the 
same  graph  for  comparison.  The  activations  are  in  different  colors,  and  if  they  both  activate  the 
same  element,  a  third  color  is  used  to  represent  this.  The  ranges  can  be  set  to  match,  or  each 
scenario  can  represent  a  different  range.  (See  Figure  13.) 


Figure  13:  Dual  Session  Comparison 
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Dual  Side  By  Side:  Dual  Side  By  Side  is  very  much  the  same  as  Dual  Session  Comparison, 
except  that  each  scenario  displays  its  own  graph.  (See  Figure  14.) 


Dual  Side  By  Side 


BBS 


[  Close  | 


Figure  14:  Dual  Side-By-Side 


4.2.8.  Graph  Manipulation 

The  graph  display  can  be  manipulated  for  better  use.  Each  element  of  the  graph  (concepts, 
edges,  and  nodes)  can  be  fdtered  out.  The  graph  nodes  can  be  moved  around,  and  the  graph 
itself  can  be  rotated  to  bring  other  nodes  to  the  front.  Nodes  can  be  centered  on  the  screen,  and 
node  size  can  be  adjusted  with  a  slider  on  the  tool  bar.  The  graph  nodes  can  be  the  concept 
nodes,  or  be  reduced  to  represent  the  concepts  themselves,  linking  to  other  concepts  that  share  a 
similar  concept  node. 

4.3  Graph  Structure 

PDAT  can  display  two  structure  types  for  analysis:  Conceptual  and  Scenario-Based. 

4.3.1.  Conceptual  Graph  Structure 

The  conceptual  graph  structure  is  generated  from  matrix  (.matrix)  files,  or  subset  Node  (.nodes) 
and  Concept  (.concepts)  files.  The  graph  is  structured  to  have  a  number  of  nodes  representing 
features  or  attributes,  and  concepts  that  create  the  edges  between  the  nodes.  Nodes  have  no 
predefined  locations  on  the  graph,  but  depending  on  the  strength  of  the  edges  between  nodes, 
they  may  be  bunched  together  in  the  graph. 
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In  the  simple  example  displayed  in  Figurel5,  there  are  two  concepts:  polar  bear  and  brown  bear. 
The  polar  bear  has  white  hair  color  (X4),  very  large  (XI 3),  long  hair  (X3),  no  tail  (XI 4),  and 
found  in  the  zoo  (XI 9).  The  brown  bear  has  brown  hair  (X9),  very  large  (XI 3),  long  hair  (X3) 
and  no  tail  (XI 4).  Attributes  that  are  exclusively  polar  bear  are  grouped  together,  and  shared 
attributes  are  also  grouped  together. 

Some  assumptions  might  be  made  looking  at  this  simple  graph.  The  edges  between  X3,  X13  and 
X14  may  be  stronger  based  on  the  fact  that  two  concepts  group  those  three  together.  When 
presented  with  a  concept  that  is  very  large  and  has  long  hair,  the  graph  may  automatically 
assume  the  concept  has  no  tail. 


O 


X4:  White  (Hair  Color) 


O 


X9:  Brown  (Hair  Color) 


3:  V  Large  (Size  Of  Body) 


X3:  Long  (Hair  Length) 


Figure  15:  Conceptual  Graph 
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4.3.2.  Scenario-Based  Graph  Structure 

The  scenario-based  graph  structure  is  built  from  a  scenario  (.sen)  file.  The  scenario  based  graph 
is  different  from  the  conceptual  graph  in  multiple  ways.  First,  there  are  multiple  types  of  nodes 
in  the  graph.  Second,  the  location  in  the  graph  is  somewhat  defined  by  the  type  of  node,  and  the 
time  in  the  scenario  when  the  node  was  generated.  Third,  all  edges  in  the  graph  are  created 
equally.  (See  Figure  16.) 


initialize  Dearborn 
'  initialize  RsshidHomc 


attended  (Asateam  AJaikum  um.| 

ralig  ion-age™ 
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attended  (Lateefa  hers.} 
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says  {Aaalaani  Aloikum  I  ar=?fc.) 
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attended  (Yes,  Haadiya.)  (2) 
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i  leaving  RashldHome 
leaving  Dearborn 


Figure  16:  Scenario-Based  Graph 
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Node  Types:  There  are  five  types  of  nodes  in  the  scenario  based  graph:  Interest,  Trigger, 
External  Event,  Decision,  and  Disposition. 

Interest:  Interest  nodes  are  spherical  nodes,  located  on  the  left  side  of  the  graph  (see  Figure  17). 
These  nodes  represent  an  interest  of  a  subject  that  will  rise  and  fall  based  on  the  scenario. 
Interest  nodes  have  thresholds,  that  when  met,  will  fire  off  the  trigger. 

Trigger:  Trigger  nodes  are  cylindrical,  located  on  the  left  side  of  the  graph,  just  to  the  right  of 
the  interest  nodes.  When  an  interest  crosses  a  threshold,  the  trigger  node  is  activated.  In  Figure 
17,  our  subject  reaches  the  threshold  for  calling  her  niece.  This  is  the  point  at  which  the  interest 
is  so  high  it  triggers  an  external  action.  The  trigger  links  to  the  external  event  that  occurs 
because  of  that  interest. 


call-fllftrp 

^-^tall.niece-abom -daughter 


SelectinjLQfltlroicce^n-concerfi-foiHlaughrer.  procedures 


Figure  17:  Interest  Trigger 

External  Event:  External  events  are  cubes  by  default  (see  Figure  18),  except  in  the  special  case 
of  a  decision  event.  External  events  provide  flow  for  the  scenario.  They  are  lined  up 
sequentially  from  top  to  bottom,  with  the  earliest  events  at  the  top.  External  events  take  up  the 
middle  sections  of  the  graph.  Within  the  middle  section  events  that  may  be  triggered  on  the  left 
side,  events  that  may  affect  disposition  are  on  the  right  side,  and  everything  else  falls  in  between. 
Typical  external  events  include  leaving  and  arriving  at  a  location,  conversation  (saying  and 
hearing),  and  picking  up  the  phone. 

Decision:  Decision  nodes  are  special  external  nodes.  Decision  nodes  are  conical  and  are  located 
in  the  middle  of  the  graph.  A  decision  node  represents  a  decision  point  for  the  subject  of  the 
graph  where  there  are  multiple  choices  for  action.  The  decision  is  influenced  by  the  subject’s 
dispositions.  The  decision  node  is  linked  to  multiple  other  external  nodes,  but  only  one  external 
node  will  be  selected.  In  Figure  18,  our  subject  is  going  to  call  her  niece  on  her  concern  for  her 
daughter.  There  are  two  choices:  calling  about  her  daughter’s  Muslim  acquaintance,  or  her 
Sunni  acquaintance. 

Disposition :  Disposition  nodes  are  spherical  nodes  on  the  right  side  of  the  graph.  Disposition 
nodes  are  attributes  used  by  decision  nodes  to  evaluate  a  situation  and  make  a  choice.  In  Exhibit 
18,  there  are  three  dispositions:  the  subject’s  religion,  the  subject’s  daughter’s  potential 
husband’s  religion,  and  the  subject’s  daughter’s  religion.  They  all  influence  the  decision.  If 
every  religion  is  the  same,  the  conversation  may  be  more  pleasant.  If  the  potential  husband’s 
religion  does  not  match  their  religion,  the  conversation  may  be  more  problematic. 
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Figure  18:  Decision 


4.4  Data  Collection  Selection 

Qualtrics  was  selected  as  the  data  collection  tool.  The  requirements  for  the  experiment  required 
a  number  of  complex  features  that  eliminated  most  of  the  other  tools.  These  requirements 
included  the  need  to  record  data  entry  and  exit  times,  auto-flipping  to  the  next  question  after  a 
predefined  amount  of  time,  and  disabling  forward  and  backward  buttons. 

4.4.1.  Category  Survey  Collection 

Qualtrics  is  being  used  to  generate  the  survey.  The  survey  is  composed  of  15  demographic 
questions  and  60  category  questions.  Demographic  questions  are  untimed,  and  the  user  can 
answer  them  at  their  own  pace.  The  category  questions  are  timed,  and  entry  times  are  recorded. 
Users  have  60  seconds  for  each  question,  with  space  for  up  to  20  categories  in  each  question. 
Between  each  category  question  was  a  black  screen  that  lasted  for  5  seconds.  The  survey  will 
take  around  75  minutes. 
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5.0 


CONCLUSIONS 


The  use  of  OMAR  to  run  scenarios  in  NWN  and  return  the  results  back  to  PD  AT  was  successful. 
The  result  displays  in  PDAT  are  very  informative,  and  will  provide  easier  analysis  of  the  data  as 
the  scenarios  become  more  complex  and  lengthy.  The  variety  of  reports  provides  a  deeper 
analysis  that  will  be  valuable  in  revealing  hidden  data. 

The  NWN  displays  do  have  their  own  issues.  The  environments  can  be  manipulated,  but  when 
running  the  scenarios,  you  are  required  to  have  an  active  response  to  the  display  in  order  to  see 
the  events  in  the  scenario.  There  have  been  discussions  on  changing  the  tool  used  for  scenario 
display,  with  OpenSim  being  a  top  candidate  to  replace  NWN.  That  tool  will  be  evaluated  in  the 
future  to  determine  if  it  has  the  functionality  to  replace  the  current  simulation  tool. 

Data  collection  will  start  in  the  near  future.  No  problems  are  anticipated  in  the  retrieval  of  the 
data.  The  tool  has  been  tested,  and  it  has  provided  few  issues.  All  issues  were  resolved  with 
simple  fixes.  Qualtrics  has  all  the  necessary  tools  for  collection  according  to  our  specific  needs, 
including  timing  and  automatic  page  change  features.  The  results  returned  from  Qualtrics  will 
be  analyzed  and  detailed  in  the  final  report. 
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LIST  OF  ACRONYMS 


2D  Two  Dimensional 

NWN  Neverwinter’s  Night 

OMAR  Operator  Model  Architecture 

PD  AT  Polyhedral  Dynamics  Analysis  Tool 

WIRTO  Warfighter  Interface  Research  and  Technology  Operations 
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LIST  OF  GLOSSARY  OF  TERMS 


Concept  An  object  made  up  of  nodes  and  edges,  can  be  identified  with  a  unique  ID. 

Typically  a  concept  will  have  a  name  (i.e.,  Beagle),  and  will  be  visible  on  the 
graph  in  its  own  color. 

Edge  An  object  that  connects  two  nodes.  In  a  concept,  all  nodes(attributes)  of  the 

concept  will  connect  to  each  other  through  an  edge.  An  edge  can  have  a  weight 
assigned  to  it,  signifying  the  strength  of  the  bond  between  the  nodes  that  it 
connects. 

Layout  A  mechanism  for  returning  (x,  y)  coordinates  for  nodes. 

Matrix  An  X  by  X  grid  of  data  that  establishes  the  connection  of  a  concept  and  its 

nodes.  The  matrix  can  also  define  concepts  and  nodes. 

Node  An  attribute  that  makes  up  a  portion  of  a  concept.  It  has  an  ID,  and  typically 

will  have  a  name  (i.e.,  Brown).  It  is  displayed  on  the  graph  as  a  filled  circle. 

•  Cut.  A  node  that  when  removed,  produces  a  graph  with  more  connected 
components  than  the  original  graph. 

•  Leaf.  A  node  that  has  one  edge 

•  Initial.  A  node  which  a  directed  edge  starts  at. 

•  Isolated.  An  unattached  node  which  has  no  edges. 

•  Normal.  A  node  that  has  both  incoming  and  outgoing  edges. 

•  Sink.  A  node  which  has  no  out  edges.  All  edges  connected  to  this  node  end 
at  this  node. 

•  Source.  A  node  which  has  no  in  edges.  All  edges  connected  to  this  node 
start  at  this  node. 

•  Terminal.  A  node  which  a  directed  edge  ends  at. 
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